---
title: Query Batching
subtitle: Receive query batches with the GraphOS Router
description: Handle multiple GraphQL requests with GraphOS Router's query batching capabilities. Aggregate operations into single HTTP requests to preserve data consistency.
---

<PlanRequired plans={["Free", "Developer", "Standard", "Enterprise"]}>

Rate limits apply on the Free plan.
Developer and Standard plans require Router v2.6.0 or later.

</PlanRequired>

Learn how query batching can help preserve data consistency of responses, and learn how to configure the GraphOS Router to receive query batches.

## About query batching 

Modern applications often require several requests to render a single page. This is usually the result of a component-based architecture where individual micro-frontends (MFE) make requests separately to fetch data relevant to them. 

Query batching makes response data consistent for MFEs making multiple requests per page. It primarily preserves data consistency rather than improve performance.

### Preserving data consistency

When the underlying data is changing rapidly, the separate requests of an application may result in responses with inconsistent data. Given two requests for a single page, the values returned in their responses may be different because the data's been updated or subgraph servers briefly had different values.

To prevent this inconsistency from happening, clients can bundle multiple requests together into a batch so routers or servers can produce responses with consistent data. MFE-based UIs usually batch multiple client operations, issued close together, into a single HTTP request. Both Apollo Client and Apollo Server support this.

### Router batching support

The router's batching support is provided by two sets of functionality:
 - client batching
 - subgraph batching

With client batching, the router accepts batched requests from a client and processes each request of a batch separately. Consequently, the router doesn't present requests to subgraphs in batch form, so subgraphs must process the requests of a batch individually. 

With subgraph batching, the router analyzes input client batch requests and issues batch requests to subgraphs. Subgraph batching is an extension to client batching and requires participating subgraphs to support batching requests. See the examples below to see illustrations of how this works in practice.

The GraphOS Router supports client and subgraph query batching.

The GraphOS Router must be configured to receive query batches, otherwise it rejects them. When processing a batch, the router deserializes and processes each operation of a batch independently. It responds to the client only after all operations of the batch have been completed. Each operation executes concurrently with respect to other operations in the batch.

### Client batching support

If you’re using Apollo Client, you can leverage the built-in support for batching to reduce the number of individual operations sent to the router.

Once configured, Apollo Client automatically combines multiple operations into a single HTTP request. The number of operations within a batch is client-configurable, including the maximum number in a batch and the maximum duration to wait for operations to accumulate before sending the batch.

## Configure client query batching

Both the GraphOS Router and client need to be configured to support query batching.

### Configure router

#### Client query batching

By default, receiving client query batches is _not_ enabled in the GraphOS Router.

To enable query batching, set the following fields in your `router.yaml` configuration file:

```yaml title="router.yaml"
batching:
  enabled: true
  mode: batch_http_link
```

| Attribute | Description | Valid Values | Default Value |
| :-- | :-- | :-- | :-- |
| `enabled` | Flag to enable reception of client query batches | boolean | `false` |
| `mode` | Supported client batching mode | `batch_http_link`:  the client uses Apollo Link and its [`BatchHttpLink`](/react/api/link/apollo-link-batch-http) link. | No Default |
| `maximum_size` | Maximum number of queries in a client batch (optional) | integer | `null` (no limit on number of queries) |

#### Subgraph query batching

If client query batching is enabled, and the router's subgraphs [support query batching](/apollo-server/api/apollo-server#allowbatchedhttprequests), then subgraph query batching can be enabled by setting the following fields in your `router.yaml` configuration file:

```yaml title="router.all_enabled.yaml"
batching:
  enabled: true
  mode: batch_http_link
  subgraph:
    # Enable batching on all subgraphs
    all:
      enabled: true
```

```yaml title="router.yaml"
batching:
  enabled: true
  mode: batch_http_link
  subgraph:
    # Disable batching on all subgraphs
    all:
      enabled: false
    # Configure(over-ride) batching support per subgraph
    subgraphs:
      subgraph_1:
        enabled: true
      subgraph_2:
        enabled: true
```

<Note>

- The router can be configured to support batching for either all subgraphs or individually enabled per subgraph.

- There are limitations on the ability of the router to preserve batches from the client request into the subgraph requests. In particular, certain forms of queries will require data to be present before they are processed. Consequently, the router will only be able to generate batches from queries which are processed which don't contain such constraints. This may result in the router issuing multiple batches or requests. 

- If [query deduplication](/router/configuration/traffic-shaping/#query-deduplication), [response caching](/router/configuration/response-caching), or [entity caching](/graphos/routing/v1/performance/caching/entity) are enabled, they do not apply to batched queries. Batching takes precedence over these features. These features still apply to non-batched queries.

</Note>

##### Example: simple subgraph batching

This example shows how the router can batch subgraph requests in the most efficient scenario, where the queries of a batch don't have required fetch constraints.

Assume the federated graph contains three subgraphs: `accounts`, `products`, and `reviews`.

The input client query to the federated graph:

```json title="simple-batch.json"
[
    {"query":"query MeQuery1 {\n  me {\n    id\n }\n}"}
    {"query":"query MeQuery2 {\n  me {\n    name\n }\n}"},
    {"query":"query MeQuery3 {\n  me {\n    id\n }\n}"}
    {"query":"query MeQuery4 {\n  me {\n    name\n }\n}"},
    {"query":"query MeQuery5 {\n  me {\n    id\n }\n}"}
    {"query":"query MeQuery6 {\n  me {\n    name\n }\n}"},
    {"query":"query MeQuery7 {\n  me {\n    id\n }\n}"}
    {"query":"query MeQuery8 {\n  me {\n    name\n }\n}"},
    {"query":"query MeQuery9 {\n  me {\n    id\n }\n}"}
    {"query":"query MeQuery10 {\n  me {\n    name\n }\n}"},
    {"query":"query MeQuery11 {\n  me {\n    id\n }\n}"}
    {"query":"query MeQuery12 {\n  me {\n    name\n }\n}"},
    {"query":"query MeQuery13 {\n  me {\n    id\n }\n}"}
    {"query":"query MeQuery14 {\n  me {\n    name\n }\n}"},
    {"query":"query MeQuery15 {\n  me {\n    id\n }\n}"}
]
```

From the input query, the router generates a set of subgraph queries:
```
"query MeQuery1__accounts__0{me{id}}",
"query MeQuery2__accounts__0{me{name}}",
"query MeQuery3__accounts__0{me{id}}",
"query MeQuery4__accounts__0{me{name}}",
"query MeQuery5__accounts__0{me{id}}",
"query MeQuery6__accounts__0{me{name}}",
"query MeQuery7__accounts__0{me{id}}",
"query MeQuery8__accounts__0{me{name}}",
"query MeQuery9__accounts__0{me{id}}",
"query MeQuery10__accounts__0{me{name}}",
"query MeQuery11__accounts__0{me{id}}",
"query MeQuery12__accounts__0{me{name}}",
"query MeQuery13__accounts__0{me{id}}",
"query MeQuery14__accounts__0{me{name}}",
"query MeQuery15__accounts__0{me{id}}",
```
All of the queries can be combined into a single batch. So instead of 15 (non-batch) subgraph fetches, the router only has to make one fetch.

| Subgraph | Fetch Count (without)| Fetch Count (with) |
|----------|----------------------|--------------------|
| accounts | 15                   | 1                  |

##### Example: complex subgraph batching

This example shows how the router might batch subgraph requests for a graph, where the client batch contains a query for an entity.

Assume the federated graph contains three subgraphs: `accounts`, `products`, and `reviews`.

The input client query to the federated graph:

```json title="federated-batch.json"
[
    {"query":"query MeQuery1 {\n  me {\n    id\n }\n}"},
    {"query":"query MeQuery2 {\n  me {\n    reviews {\n      body\n    }\n  }\n}"},
    {"query":"query MeQuery3 {\n  topProducts {\n    upc\n    reviews {\n      author {\n        name\n      }\n    }\n  }\n  me {\n    name\n  }\n}"},
    {"query":"query MeQuery4 {\n  me {\n    name\n }\n}"},
    {"query":"query MeQuery5 {\n  me {\n    id\n }\n}"}
]
```

From the input query, the router generates a set of subgraph queries:
```
"query MeQuery1__accounts__0{me{id}}",
"query MeQuery2__accounts__0{me{__typename id}}",
"query MeQuery3__products__0{topProducts{__typename upc}}",
"query MeQuery3__accounts__3{me{name}}",
"query MeQuery4__accounts__0{me{name}}",
"query MeQuery5__accounts__0{me{id}}",
"query MeQuery2__reviews__1($representations:[_Any!]!){_entities(representations:$representations){...on User{reviews{body}}}}",
"query MeQuery3__reviews__1($representations:[_Any!]!){_entities(representations:$representations){...on Product{reviews{author{__typename id}}}}}",
"query MeQuery3__accounts__2($representations:[_Any!]!){_entities(representations:$representations){...on User{name}}}",
```
The first six queries can be combined into two batches—one for `accounts` and one for `products`. They must be fetched before the final three queries can be executed individually. 

Overall, without subgraph batching, the router would make nine fetches in total across the three subgraphs, but with subgraph batching, that total is reduced to five fetches.

| Subgraph | Fetch Count (without)| Fetch Count (with) |
|----------|----------------------|--------------------|
| accounts | 6                    | 2                  |
| products | 1                    | 1                  |
| reviews  | 2                    | 2                  |

### Configure client

To enable batching in an Apollo client, configure `BatchHttpLink`. For details on implementing `BatchHttpLink`, see [batching operations](/react/api/link/apollo-link-batch-http/).

### Configuration compatibility

If the router receives a query batch from a client, and batching is *not* enabled, the router sends a `BATCHING_NOT_ENABLED` error to the client.

## Query batching and GraphOS plans
When processing batched operations, the router counts each entry in the batch as a distinct billable operation that counts towards your graph usage. Sending a batch with `N` operations is counted the same as sending `N` requests of the same non-batched operation. 

Note that GraphOS plans only track router operations, so configuring subgraph batching does not impact your graph usage count.

## Metrics for query batching

Metrics in the GraphOS Router for query batching:

<table class="field-table metrics">
  <thead>
    <tr>
      <th>Name</th>
      <th>Attributes</th>
      <th>Description</th>
    </tr>
  </thead>

<tbody>
<tr class="required">
<td style="min-width: 150px;">

##### `apollo.router.operations.batching`

</td>
<td>

mode
[subgraph]

</td>
<td>

Counter for the number of received (from client) or dispatched (to subgraph) batches.

</td>
</tr>

<tr class="required">
<td style="min-width: 150px;">

##### `apollo.router.operations.batching.size`

</td>
<td>

mode
[subgraph]

</td>
<td>

Histogram for the size of received batches.

</td>
</tr>
</tbody>
</table>

The `subgraph` attribute is optional. If the attribute isn't present, the metric identifies batches received from clients. If the attribute is present, the metric identifies batches sent to a particular subgraph.

## Query batch formats

### Request format

A query batch is an array of operations.

```graphql
[
query MyFirstQuery {
  me {
    id
  }
},
query MySecondQuery {
  me {
    name
  }
}
]
```

### Response format

Responses are provided in JSON array, with the order of responses matching the order of operations in the query batch.

```json
[
  {"data":{"me":{"id":"1"}}},
  {"data":{"me":{"name":"Ada Lovelace"}}}
]
```

## Error handling for query batching

### Batch error

If a batch of queries cannot be processed, the entire batch fails.

For example, this batch request is invalid because it has two commas to separate the constituent queries:

```graphql
[
query MyFirstQuery {
  me {
    id
  }
},,
query MySecondQuery {
  me {
    name
  }
}
]
```

As a result, the router returns an invalid batch error:

```json
{"errors":
  [
    {"message":"Invalid GraphQL request","extensions":{"details":"failed to deserialize the request body into JSON: expected value at line 1 column 54","code":"INVALID_GRAPHQL_REQUEST"}}
  ]
}
```

### Excessive queries batch error

If the number of queries provided exceeds the maximum batch size, the entire batch fails.

For example, this configuration sets a batch size limit of 2, but three queries are provided:
```yaml
batching:
  enabled: true
  mode: batch_http_link
  maximum_size: 2
```

```graphql
[
  query MyFirstQuery {
    me {
      id
    }
  },
  query MySecondQuery {
    me {
      name
    }
  },
  query MyThirdQuery {
    me {
      name
    }
  }
]
```

As a result, the router returns an error:
```json
{
  "errors": [
    {
      "message": "Invalid GraphQL request",
      "extensions": {
        "details": "Batch limits exceeded: you provided a batch with 3 entries, but the configured maximum router batch size is 2",
        "code": "BATCH_LIMIT_EXCEEDED"
      }
    }
  ]
}
```

### Individual query error

If a single query in a batch cannot be processed, this results in an individual error.

For example, the query `MyFirstQuery` is accessing a field that doesn't exist, while the rest of the batch query is valid.

```graphql
[
query MyFirstQuery {
  me {
    thisfielddoesnotexist
  }
},
query MySecondQuery {
  me {
    name
  }
}
]
```

As a result, an error is returned for the individual invalid query and the other (valid) query returns a response.

```json
[
  {"errors":
    [
      {"message":"cannot query field 'thisfielddoesnotexist' on type 'User'",
       "extensions":{"type":"User","field":"thisfielddoesnotexist","code":"INVALID_FIELD"}
      }
    ]
  },
  {"data":{"me":{"name":"Ada Lovelace"}}}
]
```

## Known limitations

### Unsupported query modes

When batching is enabled, any batch operation that results in a stream of responses is unsupported, including:
- [`@defer`](/graphos/operations/defer/)
- [subscriptions](/graphos/operations/subscriptions/)
